1. Field of the Invention
The present invention relates generally to a system and method for improving performance of a memory hierarchy in data processing. More particularly, it relates to such a system and method which improves system performance of a cache memory. Most especially, the invention relates to such a system and method which substantially reduces miss penalties for cache memory accesses.
2. Description of the Prior Art
Cache performance is becoming increasingly important since it has a dramatic effect on the performance of advanced processors. Table 1 lists some cache miss times and the effect of a miss on machine performance. Over the last decade, cycle time has been decreasing much faster than main memory access time. The average number of machine cycles per instruction has also been decreasing dramatically, especially when the transition from complex instruction set computer (CISC) machines to reduced instruction set computer (RISC) machines is included. These two effects are multiplicative and result in tremendous increases in miss cost. For example, a cache miss on a Digital Equipment Corporation VAX 11/780 only costs 60% of the average instruction execution. Thus, even if every instruction had a cache miss, the machine performance would slow down by only 60%. However, if a RISC machine like the Digital Equipment Corporation WRL Titan has a miss, the cost is almost ten instruction times. Moreover, these trends seem to be continuing, especially the increasing ratio of memory access time to machine cycle time. In the future, a cache miss all the way to main memory on a superscalar machine executing two instructions per cycle could cost well over 100 instruction times. Even with careful application of well-known cache design techniques, machines with main memory latencies of over 100 instruction times can easily lose over half of their potential performance to the memory hierarchy. This makes both hardware and software research on advanced memory hierarchies increasingly important.
TABLE 1 ______________________________________ cycles cycle mem miss miss per time time cost cost Machine instr (ns) (ns) (cycles) (instr) ______________________________________ VAX 11780 10.0 200 1200 6 .6 WRL Titan 1.4 45 540 12 8.6 ? 0.5 4 280 70 140.0 ______________________________________